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ABSTRACT 

This paper describes the notation and underlying 
organization of ai' augmented, recursive-transition network grammar 
and illustrates how such a grammar is a natural medium for expressing 
and explaining a wide variety of facts about the psychological 
processes of sentence comprehension. A general discussion of 
transformational grammar and psycholinguistics serves as background 
information. Details on the structure, characteristics, and operation 
of the augmented, recursive- transition network grammar are provided. 
There is a discussion concerning the formalization of perceptual 
strategies and how these strategies can be naturally represented in 
transition network grammars. A concluding section explains the 
justification of transition network models. A list of references is 
included, and diagrams illustrate the working of the grammar in 
several problem areas. (VM) 
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I. Introduction 



Durinr; the pa.st year a ma.ior research effort has been 
conducted to explore and refine the properties of an aup;mented 
recursive transition netv;ork parser (.V/oods, 1969, 1970) and to 
develop a large-scale English grammar for the system. Although 
our primary goal has been to construct a powerful and practical 
natural language processor for artificial intelligence and 
information retrieval applications, we have also investigated 
the correspondence between the sentence processing characteristics 
of the parser and those of human speakers, as revealed by 
psychological experimentation, observation, and intuition. We 
have found that the grammatical formalism of the transition 
netv/ork Is a convenient and natural notatlonal system for fabri- 
cating psychological models of syntactic analysis . In the present 
paper we describe some of the psychologically appealing properties 
of the parser, and illustrate how psychollngulstlc experimental 
results can be mapped into simple transition network models. 

V/e suggest that building and testing such models can lead to a 
better understanding of linguistic performance. 

It should be clear from the outset that we are not proposing 
a transition network model as a complete and sufficient represen- 
tation of all aspects of language behavior. Rather, transition 
network models aim only at simulating the syntactic analysis 
component of performance: given an Input string written In 
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standard orthography, they attemnt to discover the syntactic 
relationships holdin.i^ between constituents. We if;:nore the myriad 
problems of phonetic decodlnir and sep;mentat ion and semantic and 
coFinltive interpretation, as well as all the psychollnp^uls tic 
and motivational comnlexltles of speech production. It is in 
this limited sense that we refer to transition netv;ork grammars 
as sentence comprehension or perceptual models. Of course, we 
expect that more complete formalizations of languapce behavior 
will Incorporate such Independently developed syntactic analysis 
models . 

In section II of this paper we sketch the lin.p;uistic and 
psycholinpUistic backptround of our research. Section III 
describes the or,p;anlzation and operation of the transition network 
oarser and depicts the p:rainmatical notation, and section IV shows 
the renresentation in this notation of perceptual strate,c|;les 
induced from psychollngulstlc data. In section V we discuss 
the fruitfulness of this modeling approach, indicating 
some conceptual issues that are clarified and some emnlrlcal 
predictions that arise from transition network formulations. 








II. Transformational Grammar and Psycholinguistics 

The process by which a native speaker comprehends and produces 
meaningful sentences in his language is extremely comnlex and, 
with our present body of psycholingulstlc theory and data, is 
understood only slightly. This shortcoming of psycholinguistics 
exists despite the fact that advances in linguistic theory over 
the last decade have provided a number of crucial insights into 
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the formal structure of lanc^uaf^e a.nd linguistic performance. 

To place augmented recursive transition netv/ork jgrannars in the 
context of previous research, v/e briefly survey some relevant 
results of linguistics and psycholinguistics . 

A transformational grammar for a given language L formally 
oefines the notion sentence of L by describing a mechanical 
procedure for enumerating all and only the well-formed sentences 
of L. With each sentence it also associates a structural des- 
cription which provides t. formal account of the native sneaker's 
competence, the linguistic knowledge which underlies his ability 
to make judgments about the basic grammatical relations (e.g., 
subject, predicate, object) and about such sentential properties 
as relative grammaticality , ambiguity, and synonym.v. At present 
there is no clear agreement among linguists about the detailed 
features required for an adequate grammar, but certain principles 
of grammar organization are almost universally accepted: the 

structural description furnished for a sentence by the grammar 
must consist of (at least) two levels of syntactic representation 

(P-markers) — a deep structure and a surface structure together 

with a specification of an ordered seo_uence of transformations 
which maps the deep structure of a sentence into appropriate 
surface structures. 

Transformational theorists maintain that their formal 
model Is not Intended to give and should not be Interpreted 
as giving an accurate account of the psychological processes 
Involved when a human being uses language, either speaking or 
comprehending. Any correlations observed between actual behavior 
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characteristics and transformational grammars are essentially 
accidental, slp;nlfylng at the very most merely the fact that 
psycholop^lcal and linguistic data are both obtained from the 
same class of native speakers (but Chomsky (1965) weakens this 
assertion somewhat when he argues that acquisition data might 
have a bearing on the evaluation metric selected for grammars). 
Linguists have been very careful to distinguish the sneaker's 
comnetence, which transformational grammars attempt to model, 
from his performance, the manner In which he utilizes his knowledge 
In processing sentences (Miller and Chomsky, 1963* Chomsky, 1965 1 
Jackendoff and Cullcover, 1970). Thus a transformational 
grammar might be allowed to generate • sentences which are virtually 
Impossible for a sneaker to deal with. Most current grammars 
will generate (5a), assigning It the same subJect-verb-obJect 
relations as are apparent In (5b): ' 

(5) (a) The man the girl the cat the dog bit scratched 

loved ate chocolate Ice cream. 

(b) The dog bit the cat that scratched the girl who 
loved the man who ate chocolate Ice cream. 

Very few native speakers would Intuit that (5a) Is grammatical, 

yet to prevent Its generation, either the grammar must be 

greatly complicated or other sentences whlcii native sneakers do 

accept must be marked ungrammatical. Linguists resolve this 

dilemma and preserve the slmnllclty and generality of their 

grammars by claiming that native English sneakers do have the 

basic knowledge to process (5a), which Is therefore grammatical: 

speakers have trouble with It because their nerceptual 

mechanisms do not provide the memory space and/or comnutatlonal 
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routines required to process it. A transformational p;rammar is 
a formal specification of the speaker’s competence and has 
nothing to say about psychological functioning. 

Despite these disclaimers, psycholinguists have been intrigued 
by transformational theory because it provides the most intricate 
and compelling explication to date of a large number of basic 
linguistic intuitions. Many experiments have been conducted to 
test the hypothesis that transformational operations will have 
direct, observable reflexes in psychological processing; Fodor 
and Garrett (I966) and Bever ( 1970 ) present useful reviews of 
this literature. A major concern of these studies has been 
to determine whether the perceptual complexity of sentences 
(the difficulty of comprehending and responding to them) is 
directly correlated wlwh derivationa l complexity (e.g., the 
number of transformations required to generate them). Fodor 
and Garrett (I966) examine this "derivational theory of complexl'ty" 
in detail and conclude that the available psychollngulstic data 
do not offer much support for it and that the connection, if 
there is one, between the transformational grammar and perceotlon 
is not very direct at all. 

Although psycholinguists have abandoned for the most part 
their attempts to find perceptual reflexes of specific grammatical 
features, a. number of studies have been successful in corrobor ating 
the psychological reality of the deep structure-surface structure 
distinction. MacKay and Bever (1967) found that subjects respond 
differently to deep structure and surface structure ambiguities; 
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V/anner (1968) shov/ed that the number of deen structure S-nodes 
underlying a sentence has a direct Influence on the ease of 
prompted recall from long-term memory; and Bever (1970) has 
reinterpreted the results of the click experiments (e.g., Fodor 
and Bever (1965 )) as demonstrating that deep structure F.-nodes 
affect the surface segmentation of a stimulus sentence. Thdse 
experiments suggest that an adequate model of sentence compre- 
hension must Incorporate some mechanism for recovering a. deep- 
Gtructure-llke representation of a given stimulus word string. 

This representation should explicitly denote at least such basic 
grammatical relationships as actor, verb, and object. More 
extensive empirical work should Indicate whether deen structure 
must be even more abstract than this. 

There are several otlier requirements for' adequacy that we 
iriay Impose on potential models of sentence comprehension, 
based on some common observations about our sentence processing 
abilities : 

(a) A perceptual model must process strings In essentially 
temporal or linear order, for this Is the order In 
which sentences are encountered In conversation and 
reading . 

(b) It must process strings and provide appropriate analyses 
In an amount of time proportional to that required by 
human speakers. For example, since perceptual difficulty 
does not rapidly Increase as the length of the 
sentence Increases, the amount of time required hy 

the model should be at most a slowly Increasing function 
of sentence length. ^ n 
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(c) The model should discover anomalies and ambiguities 
where real speakers discover them, and for ambiguous 
sentences the model should return analyses In the 
same order as speakers do. 

V/hereas there are many well-known recognition procedures 
for programming languages and other relatively slmole artificial 
languages, only a few algorithms have been proposed which aim 
at "transformational" recognition, that Is, which attemot to 
develop appropriate deep structures from natural language 
surface strings. Some of these algorithms (Matthews, 1962: Petrlck, 
1965.; Zwlcky et al. 1965) Incorporate more or less directly 
a linguistically motivated transformational grammar; In light 
of the empirical shortcomings of the derivational theory of 
complexity. It Is not surprising that these proposals are Inadequate 
perceptual models (for detailed psychological and formal critiques 
see V/anner (I968) and Kelly (1970)). A deep structure recovery 
strategy suggested by Kuno (1965) operates Independently of a 
transformational grammar and offers more psychological relevance, 
but it too has formal limitations (Kelly, 1970). A procedure and 
grammatical notation recently described by Kaplan (1970), based 
on an algorithm by Kay (1967), appear to meet many of the formal 
and practical requirements for deep structure recovery, but 
at present not enough Is knovm about Its operating characteristics 
to assess Its adequacy as a formalism for perceptual models. 
Augmented recursive transition network grammars, to which we 
now turn, can satisfy (a) - (c), have other desirable psycho- 
logical and formal properties, and have the additional advantage 
of being practical and efficient. 
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III. The Augmented Recursive Transition Network 

The idea of a transition network parsing procedure for natural 
language was originally suggested by Thorne, Brat ley, and Dewar 
(1968) and was subsequently refined in an implementation by 
Bobrow and Fraser (1969) • Woods (1969* 1970) has also presented 
a transition network parsing system which is more general than 
either the Thorne et al . or Bobrow-Praser systems. The discussion 
below is based on the Woods version. Since a detailed descrln- 
tlon is already available (Woods, 1969, 1970), we nresent here 
only a brief outline of the grammatical formalism and then focus 
on the manner in which this formalism can be used to express 
perceptual and linp;ulstlc regularities. 

At the heart of the augmented recursive transition network 
is a familiar finite-state grammar (Chomsky, 1963) consisting 
of a finite set of nodes (states) connected by labelled directed 
arcs. An arc represents an allowable transition from the state 
at its tall to the state at its head, the label Indicating; the 
input symbol which must be found in order for the transition to 
occur. An input string is accepted by the grammar if there 
is a path of transitions which correspond to the sequence of 
symbols in the string and which lead from a specified initial 
state to one of a set of specified final states. Finite state 
grammars are attractive from the perceptual point of 
vlev; because they process strings strictly from left to right, 
but they have well-known inadequacies as models for natural 
languages (Chomsky, 1957)* "f^or examnle, they have no machinery 
for expressing statements abput hierarchical structure. 

. 8 
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This particular weakness can be eliminated by adding a 
recursive control mechanism to the basic strategy, as follows: 
all states are given names which are then allowed as labels on 
arcs in addition to the normal input-symbol labels.', When an 
arc v/ith a state-name is encountered, the name of the state at 
the head of the arc is pushed (saved on the top of a push-down 
store, and analysis of the remainder of the input string, 
continues at the state named on the arc. When a final state 
is reached in this new part of the "grammar , a pop occurs (control 
is returned to the state removed from the top of the push-down 
store). A sentence is said to be accepted when a final state, 
the end of the string, and an empty push-down store are all 
reached at the same time. Note that with this elaboration of 
the basic finite-state mechanism, we have produced a formalism 
that can easily describe context-free languages as well as 
regular languages with unbounded coordinate structures. The 
structural description provided for a sentence by the procedure 
is simply the history of transitions, pushes, and pops required 
to get through the string. 

However, the finite-state transition network with recursion 
cannot describe cross-serial dependencies, so it is still Inadequate 
for natural languages (Postal, 196^a). The necessary additional 
power is obtained by permitting a sequence of actions and a 
condition to be specified on each arc. The actions provide a 
facility for explicitly building and naming tree structures. 

The names, called registers , function much like symbolic variables 
in programming languages: they can be used in later actions, 

perhaps on subsequent arcs, to refer to their associated 
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structures. A register is said to contain the structure it names, 
and the actions determine additions and changes to the contents 
of registers in terms of the current inout symbol, the previous 
contents of registers, and the results of lov/er-level comnutations 
(pushes). This means that as constituents of a sentence are 
identified, they can be held in registers until they are combined 
into larger constituents in other registers. In this way 
deep structural descriptions can be fashioned in registers 
essentially independently of the analysis paths through the 
transition network. 

Conditions furnish more sensitive controls on the admissibility 
of transitions . A condition is a Boolean combination of predicates 
involving the current input symbol and register contents. An 
arc cannot be taken if its condition evalutes to false (symbolized 
by NIL), even though the current input symbol satisfies the arc 
label. This means first, that more elaborate restrictions can 
be imposed on the current symbol than those conveyed by the arc 
label, and second, that information about previous states and 
structures can be passed along in the netvrork to determine 
future transitions. This makes it possible for similar sections 
of separate analysis paths to be merged for awhile and then 
separated again — a pov/erful technique for eliminating 
redundancies and simplifying grammars. The condition predicates 
and the arc actions can be arbitrary functions in LISP notation, 
although we have developed a small set of primitive operations , 
described below and in Woods (1969, 1970), which seem adequate 
for most situations. In these primitive actions and pppdicates., 
atomic arguments denote reglstersj parenthetic expressions are 
forms to be evaluated. • it 10 
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In order to be able to refer to the current Input symbol 
In conditions or actions, a special register, named *, has been 
provided. More properly, this register always contains the 

constituent that enabled the transition: usually this is the 
input symbol, but for actions on a push arc (which are usually 
executed after the return from the lower level), * contains the 
structural description of the phrase identified in the lower 
computation. This phrase is determined when a special type of 
arc, a pop arc, is taken from a final state at the lower level 
(final states are distinguished by the existence of pop arcs). 

The recursive transition network, with all of these additions 
is called an augmented recursive transition network: it is 
easy to show that it has the generative power of a Turing machine 
To demonstrate more concretely how the transition netvrork works, 
’we give a simple example. Figure 1 shows a transition network 
grammar that v/lll recover deep structures for simple transitive 
and intransitive sentences, such as (6) and (7): 

(6) The man kicked the ball. 

(7) The ball fell. 

The top of the figure shows the organization of paths in the 
netv;ork. States are represented by circles with the state name 
inside. The state-names are purely mnemonic, serving to indicate 
the constituent being analyzed (to the left of the slash) and 
how much of. that constituent has been identified so far. Each 
arc specifies what will allow the transition and has a number 
denoting the condition and actions in the table belov;. We 
mentioned above three kinds of arcs: ordinary input symbol arcs. 
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push arcs, and pop arcs. To distinguish these arcs from each 

other and from other arc types, each arc has an explicit tyoe 

indicator. Thus, PUSH NP/ specifies that arc 1 is a push 

arc and that control is to pass to state NP/. POP (SBUILD) indicate 

that arc 5 Is a pop arc, and the structure to be popped (that 

is, placed in * at the next higher level) is the value of the 

function SBUILD. Figure 1 includes two new types: a CAT 

arc (arc 2) does not require a specific input symbol, but 

requires that the v/ord be marked in the dictionary as belonging 

to the specified lexical category. A JUMP arc (arc is a 

very special arc that allov;s a transition in the grammar, with 

possible actions, without advancing the input string — it is 

useful for bypassing optional grammar elements. 

Let us trace the analysis of sentence (6) using this grammar 
(Figure 2 shov/s the trace as it is printed out by the program. ) 

The starting state is, by convention, the state labelled S/ . 

The only arc leaving S/ is a push for a noun-phrase, so without 
advancing the input string, we switch to NP/. Since the , the 
current input symbol, is in the category DET and since the 
condition for arc 6 is trivially true, we can take arc 6, 
executing the action (SETR DET *). SETR is a nrimitive action 
that places the structure specified by its second argument 
(in this case, the current input word, denoted by *) in the 
register named by its first argument (DET) . Thus after following 
arc 6, the register DET contains the , and we continue processing 
at state NP/DET., looking at the word man . Vie are permitted to 
take arc 7, saving man in the register N, and arrive at the final 
state NP/N. We take the POP arc, which defines the phrase to 

12 
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be returned. BUILDQ is a primitive action that takes as its 
first argument a tree fragment with some nodes denoted by the 
symbol +. These nodes are replaced by- the contents of the 
registers specified as the remaining arpcuments, in left-to-right 

order. Thua the value returned by arc 8 will be the structure 

' (NP (DET the ) (N man ) ) . which is a labelled bracketing cor- 
responding to the tree (8); 



This structure is returned in the register * on arc 1, v/here 
the action (SETR SUBJ *) places it in the register SUBJ. We 
move on to state S/SUBJ, looking at the word kick . 

Kick satisfies the label on arc 2, so the condition is 
evaluated, checking the inflectional features in the dictionary 
entry for kick . The predicate (GETF TSN) verifies that the verb 
is a tensed form (as opposed to a partlcinle) , and SVAGR ascer- 
tains that the person-number code of the verb agrees with the 
noun-phrase stored in the register SUBJ. Since the condition 
is true, the transition is permitted and the action is executed. 
Setting the register TNS to the value of the feature TNS (in 
this case it would be PAST) and saving the verb in V. At state 
VP/V, we have a choice of two arcs. Arc 3 Is a push for an 
object noun-phrase, which we can take since (TRANS V) is true, 
that is, since the verb in V is marked transitive In the dictionary. 
We execute the push. Identify the noun-phrase the ball, and save 
it in the register OBJ. At S/VP we ' pop the value of SBUILD 
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a function which applies a complicated BUILDQ to the 
re?!:lsters SUBJ , TNS, V, OBJ, building the tree (9). Notice that 
at this point we have exhausted the input string, achieved a 
final state, and emptied the push-down stack. Thus the sentence 
({]) is accepted ’ey the grammar, and its deep structure is the 
structure returned by the final POP. 
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the ball 




Sentence (7) is processed in the sane way, except that arc ^ is 
taken instead of arc 3, since fell is marked intransitive. Hence, 
tile resulting structure does not have the object NP node. 

For these two examples and, indeed, for all sentences in the 
language of this grammar, the structure returned by the final 
POP directly reflects history of the analysis, the surface 
structure, but this need not be the case. As a second illustration, 
we extend tlie grammar to deal with passive sentences, such 



as (10) : 

(10) The ball was kicked by the man. 

We must add one new state, S/BY, a new arc to state VP/V and two 
new arcs to state S/VP. In addition, we must change the conditions 

on arcs M and 5. Figure 3 shows the new grammar, with new arcs 
In boldface and with only new and changed conditions and actions. 
For sentence (10) the new grammar works as follows; the ball is 
recognized as a noun-phrase and placed in SUBJ . Was passes the 
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condition on arc 2, so PAST Is stored In TNS and Is placed In 
V (as part of the category checking operation, the Inflected form 
was is replaced In * by Its root). At this point in the sentence, 
v;e do not know If ^ Is a passive marker or a main verb as in (11) 

(11) The ball was a sphere. 

V/e make the assumption that it Is a main verb, v;lth the under- 
standing that later Information might cause us to change our 
minds and possibly rearrange the structure we have built. At 
state VP/V we find that we have Indeed made a mistake. We first 
attempt the arc 9 transition. We are looking at kicked , the past 
participle of a passlvlzable verb, and ^ is in V, so we can make 
the transition: the contents of SUBJ ( the ball) are moved to 

OBJ and SUBJ Is emptied (a register containing NIL is considered 
void). Then kids replaces ^ In V, and we re-enter state VP/V, 
looking at the word by . 

By is not a verb, so arc 9 is disallowed. Kick Is transitive 
so we try pushing for a noun-phrase, but since ^ is not a 
determiner, the push is unsuccessful. Arc 4 has been modified 
so that It can be taken If the verb is transitive but the object 
register has already been filled (the predicate PULLR is true 
Just in case the Indicated register is non-empty, and we can 
therefore JUMP to S/VP. 

At S/VP we cannot take arc 5 because we have no subject, 
so we try arc 10, a WRD arc. This arc type corresponds to the 
original finite-state grammar arc label, a symbol which must 
literally match an input word. Arc 10 specifies WRD BY and 
matches the current word, so the transition Is allowed (NULLR 
is true when FULLR Is NIL) . 
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At this point In the sentence, the only way we could not 
have a subject Is If we had followed the nasslve loop. V/e therefore 
look for the deep subject of the sentence In a by-phrase: we 
take arc 11, put the man In SUBJ , and return to S/VP , from v/hlch 
v;e pop* 'fhe resultlnp; structure Is Identical to (9) — we have 
undone the nasslve transformation. If the a°:ent phrase had been 
omitted In (10), we v;ould have taken arc 12 Instead of the path 
through S/DY. Arc 12 Is a JUMP that Inserts the pronoun 
someon e In SUDJ Just In case there Is no other way to p;et a 
subj ect . 

These simple examples have Illustrated the notation and 
underlying organization of the augmented recursive transition 
network. They have also demonstrated that transition 
network grammars can perform such transformational operations 
as movement, deletion, and Insertion In a stralghtforviard manner. 

We are now ready to examine the way In which transition network 
grammars can model performance data. 

IV. The Formalization of Perceptual Strategies 

Dever (1970) has surveyed the results of many psychollngulstlc 
experiments and has Inferred from the data that human belrifTS use 
a small number of perceptual strategies In processing sentences. 
.Some of these are corollaries of more general cognitive strategies 
and have observable reflexes In other areas of perception, 
while others are peculiar to language performance. As a set, 
these strategies account In part for the relative perceptual 



-19- 



Kanlan 



complexity of sentences and for some of the patterns of 
observed perceptual errors. In this section, we show hov; these 
strategies can be naturally represented in transition netv/ork 
grammars . 

The dependent variable in a majority of psycholinguistic 
studies has been the difficulty subjects experience in nrocessinr 
sentences, as indicated for example by response latencies, 
recall errors, and the impact of various disturbances on compre- 
hensibility. Thus the ultimate validation of transition network 
models will depend to a large extent on the correlation betv/cen 
experimentally observed complexity and complexity as measured in 
the transition netv;ork. There are several ways of defining a 
complexity metric on the netv;ork. We could count the total 
number of transitions taken in analyzing a sentence, the total 
number of structure building actions executed, pr even the total 
number of tree-nodes built by these actions. We could also use 
the amount of memory space or computing time required for a sentence 
in a particular inn lamentation of the transition network narser 
(e.g., the number of conses (memory cells) or seconds indicated 
in Figure 2). Of course, most intuitive measures of complexity 
are highly intercorrelated and lead to the same predictions, so 
our choice can be somewhat arbitrary. V^e will say that the 
complexity of a sentence is directly proportional to the number 
of transitions made or attempted during the course of its analysis. 

With this definition the* complexity of a sentence depends . 
crucially on the order in which the network is searched for a 
successful path, although its acceptability by the grammar is ' 
independent of the search-order. Unless special mechanisms are 
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Invoked, the arcs leaving a state-circle are tried in clockwise 
order, starting from the top. Thus in Figure 3, arc 5 is 
attempted before arcs 10 and 12. If an attempted arc turns out 
to be permitted, then the remaining, untried arcs leaving the 
state are held in abeyance on a list of alternatives, and the 
legal transition is made. If the path taken is subsequently 
blocked, alternatives are removed from the front of the list and 
tried until another legal path is found. As a result of this 
depth-first search, an ambiguous sentence will initially provide 
only one analysis; the other analyses are obtained by simulating 
blocked paths after successes. 

A. The Relations Between Clauses 

Since sentences are frequently composed of more than one 
clause, the native speaker must have a strategy for deciding how 
the component clauses of a sentence are related to each other 
(e.g., which is the main clause, v;hich are relative clauses, 
and which are subordinate). Bever nronounds that "the first N..V..(N) 
clause... is the main clause, unless the verb is marked as sub- 
ordinate" (Ibid, Strategy B, p. 29^), and points out that a sentence 
is perceptually more complicated whenever the first verb is not 
the main verb, even if it is marked as subordinate. Thus, sentences 
with preposed subordinate clauses (12b) are, according to this 
hypothesis, relatively more difficult than their normally ordered 
counterparts (12a); 

(12) (a) The dog bit the cat because the food was gone. 

(b) Because the food was gone, the dog bit the cat. 
(sBever’s (2^a-b)) 
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And In cases where the verb Is subordinate but not marked as 

such, this strategy can lead to serious perceptual errors. 

Bever reports that subjects had much more difficulty understanding 

sentences like (13a), where there is an Illusory main verb and 

sentence (underlined), than (13b), even though both sentences, 

being: center-embedded, are exceedingly difficult: 

(13) (a) The editor authors the newspaper hired liked 

laughed. 

(b) The editor the authors the newspaper hired 
liked laughed. (»Bever’s (27a-b)) 

The modifications to our transition network shown in Figure 

^ can account for these facts. We have added two arcs at the 

S level to look for subordinate clauses; a simple transition sequence 

(not shown) analyzes and builds the appropriate structure for them. 

Also, we have expanded states NP/ to allow null determiners, and 

NP/N to look for relative clauses. With this grammar, four more 

arcs, 1, 6, 17, and 7, must be attempted for (12b) than for (12a). 

For (12b), first arc 1 is tried, causing a push to NP/ where arcs 

6, 17, and 7 are tried and fall. We back up to state S/ and take 

arc 13, eventually ending up with the appropriate structure 

(the complete sequence of attemoted arcs is 1, 6, 17, 7, 13, 

SUDORD/ arcs (not shown), 1, 6, 7, 8, 2, 9, 3, 6, 7, 8, 1^, 5). 

Note that we must still attempt arc 1^, even though we know the 

condition will fall, because it is ordered before the pop arc, arc 

5. For (12a), our first try at arc 1 takes us straight through 
to arc 1^, where we pick up the subordinate clause, consider arc 
1^1 again, and then pop at arc 5 (sequence = 1, 6, 7, 8, 2, 9, 3, 

6, 7, 8, li|, SUBORD/arcs, 1^1, 5). 

The difference between (13a) and (13b) is equally well 
accounted for. Arc 15 looks for a relative clause on the noun- 
phrase, given that there is a relative pronoun following the 
. The arc has two new actions, SENDR and ADDR. Registers 



noun 
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are subiect to the control of the rush-down recursion meohar.lsn, 
so that when a rush Is executed, the retrlsters' contents at 
the urner level are saved on the stack alonr, with the actions 
to be executed upon return, and at entry to the lower-level, the 
registers are all enotv. Upon roPPlhf.. ’^he upper level registers 
are. restored. SENDE is a very special action: it can onlv arroar 

on .1 PUSH arc, and It is the only action executed before rushlny. 

It causes structures computed at the unper level to be placed in 
registers at the lower level. Thus the action CSENDH V-H (KPIiUILD)) 
causes the noun-nhrase so far identified to be placed in the 
WH register at state R/, the beginning of the relative clause network 
(not shown). Based on the Internal structure of the relative clause, 
the R/ network then decides whether the relativised noun-phrase in 
WH is to be Interpreted as the subject or object, analyzes the 
clause using parts of the S/ and HP/ networks, and returns the appro- 
priate structure. (ADDR PEL *) causes this structure to be added on 
the right of the previous contents of EEL, so that a sequence of 
relative clauses can be processed by looping through arc 15. In 
(13a-b), however, there is no relative pronoun, so we cannot take arc 
15. For both sentences, a successful analysis requires that we oush 
to state R/NIL (arc l6) , the section of the relative clause grammar 
designed to analyze relatives with missing relative pronouns, 
before we get to arc l6, we pop via arc 8 to S/SUBJ. In Cl3a), the 
input word at this point is authors , a possible verb, so we can take 
arc 2 to state VP/V. We continue on until we try to poo at arc 5 
without having consumed the input string (the current word is hli^d) 
and by the time we have backed up all the way to the appropriate arc 
16, we have attempted seventeen arcs erroneously. For (13b), since 
thl is not a verb, we are blocked at state S/SUBJ, and we arrive 
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at arc l6 having only attempted three arcs. Inside the relative 
clause grammar, the noun phrase authors In (13a) requires an extra 

transition at arc 17, so the net difference between the two 

sentences Is fifteen arcs not counting the blocked R/NIL arcs In 

[13a], a difference clearly In line with empirical perceptual complexity. 

V/e have thus expanded our simple grammar to accept and 
provide deep structures for a variety of constructions. Our 

grammar has the same formal power to describe these structures 
as a transformational grammar, but v/e have been able to arrange 
the analysis path so that complexity In our model corresponds to 
perceptual complexity, as stated by Bever's Strategy B. We 
have taken advantage of the fact that, unlike the ordering of 
transformations, the order of arcs can be freely changed, 
radically altering the amount of computation required for particular 
sentences, without affecting the class of acceptable sentences. 

B. Functional Labels 

A major task In sentence comprehension Is the determination 
of the functional relationships of constituents within a single 
clause, of deciding who the subject Is, what the action Is., etc. 

Bever suggests a simple strategy for assigning functional labels 
based on the left-to-rlght surface order of constituent s : "Any 

Woun-Verb-Noun (NVN) sequence within a potential Internal [deep 
structure] unit In the surface, structure corresponds to ’actor- 

ij. 

action -object'" (Ibid. Strategy D, p. 298). Bever cites several 
perceptual studies Involving sentences for. which this strategy 
Is misleading, and In all cases, these sentences were more 
difficult to respond to than control sentences for which strategy 
D was. appropriate. 
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There Is very good evidence that passive sentences are more 
difficult to process than corresponding actives. In the absence 
of strong semantic constraints. Given strategy D, this follows 
from the fact that the surface order of passives Is object-action- 
actor. Similarly, progressives (l4a) have been found to be 
significantly easier to comprehend than superficially Identical 
participial constructions (l^b) (Mehler and Carey, 1968). 

(14) (a) They are fixing benches. 

(b) They are performing monkeys. 

(= Bever’s (31a-b)) 

According to strategy D, performing Is Initially accepted as the ^ 
main verb, until the spurious direct object monkeys^ is encountered; 
at this point the labels must be switched around. 

Bever explains these processing difficulties In terms of the 
amount of re-labellng that is required, given that strategy D 
can lead to errors. This translates Into the proposition that 
relative complexity Is measured by the degree to which constituents 
are shifted In registers, since assigning a constituent to a 
register Is the transition network analog of functional labeling. 
Indeed, Figure 3 shows that SUBJ Is reset twice more for passives 
than for actives, while In Figure 5 participial sentences require one 

extra register assignment (NMODS). However, we have defined com- 
plexity In terms of the number of arcs attempted, and we now show 
that this measure can also account for the experimental results. 

Figure 3 contains the arcs necessary for passive sentences . 
Simple active (6) and passive ( 10 ) sentences are treated 
Identically until state VP/V Is reached. Arc 9 Is attempted for 
both of them and Is taken for the passive, returning to VP/V. 

9 Is attempted again but falls,,^ and then twelve additional arcs . 
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are tried before the successful final pop is executed. Since only 
six additional arcs are attempted for the active, the difference 
in favor of the relative complexity of the passive is six. The 
difference is seven for the more complicated grammar in Figure 5. 

Figure 5 gives the necessary modifications for the progressive 
and participial constructions. Arc 18 can be taken only if the 
current word is a present participle and the previously identified 
main verb is The actions put the new verb in V and mark TNS 

as progressive. Arc 19 simply adds an identified participle to 
NMODS, where the function NPBUILD will find it. The analysis of 
(l^a) is simple: At state VP/V, the current word will be 

fixing and ^ will be in V, so that arc l8 can be taken. Since 
fix is transitive, benches will be identified as the direct object, 
and the pop at arc 5 will be successful. (l^b) involves considerably 
more effort. At VP/V, arc 18 will also be taken but arc 3 is ruled 
out with perform in V (see- footnote M . Before returning to arc 3 
with ^ in V, arcs 1^, 5, 10, and 12 will be tried, and additional 
arcs will be attempted in deriving the correct participial 
analysis (we assume that is marked transitive). 

Thus the functional re-labelling and the attempted transitions 
explanations account equally v/ell for the experimental observations. 
At present we have no firm empirical basis for choosing one com- 
plexity measure over the other; we must find crucial sentences where 
the measures make opposing predictions and let the data decide for 
us. So. far, we have been unable to discover such sentences . 
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C. Prenomlnal Adjective Orderinp; 

I 

Another problem concerns the sep;mentatlon of superficial 
sequences of words into structural units. Where does a noun-phrase 
begin, for example, and;where does it end? That these are not 
trivial questions is illustrated by (15a-b), where the role of 
marks is unclear until the v;hole sentence has been processed. 

(15) (a) The plastic pencil marks easily. 

(b) The plastic pencil marks were up;ly. 

(= Bever’s (66a-b)) 

Of course, no matter v;hat perceptual stratef^y is involved in 
makinp; these decisions, the transition network parser will continue 
trying alternative paths until it arrives at the correct segmen- 
tation, but an appropriate strategy would make the analysis more 
efficient. Bever suggests that in recognizing the end of a noun- 
phrase, native speakers use a strategy which also accounts for 
the anomalies in such pairs as (without contrastive stress): 

(16) (a) The red plastic box... 

(b) *The plastic red box... 

(c) The large red box... 

(d) *The red large box... 

(= Bever’s (6?a-d)) 

He cites the theories of Martin (19^8) and Vendler (1968) vxhlch 

5 

essentially claim that the more ’’nounlike" an adjective is, the 

closer to the noun it must be placed. Thus the anomalies in 

(l6) are accounted for if v;e assume that plastic is m.ore nounlike 

than red and red is more nounlike than large . Although the notion 

nounlke is not made very precise, Bever gives heuristic 

arguments that these assumptions are correct. He then postulates 



o 
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that the end of a noun phrase Is sisnalled by a word v;hlch 
Is less nounlike than preceding words ( Ibid . , Strategy E” , p. 323). 
Since large Is less nounlike than red, the Initial noun phrase 
In (l6d) must be the red . 

This constraint Is difficult to express In traditional 
transformational formalisms but is quite directly representable 
in the transition network. It not only makes the transition 
network more conguent with performance data, but also helps to 
rule out the anomalies In (l6). Assuming that nounlike is vrell- 
deflned and that all potential nouns (Including adjectives )' are 
In category N and have their nounllke-ness marked In. the lexicon, 
Figure 6. shows the necessary addition to the network. Arc 20 
Is attempted before the pop from NP/N. If the nounllke-ness of 
the current word Is greater than or equal to that of the word In 
N, then the word In N Is not the head of the ncun-phrase. V/e add 
this word to the list of modifiers in NMODS, and place the current 
word in N, as a new candidate for head noun. We continue looping 
until we find a word that Is less nounlike than the head, marking 
the end of the noun-phrase. This procedure will accept (l6a,c) 
but reject (l6b,c) except in constructions along the lines of 
(17) . In (17) the adjectives are accepted only because they can 
be analyzed In separate noun-phrases: 

(17) I like the plastic red boxes are made of. 



- 28 - 



Kaplan 



V. The Justification of Transition Network Models 

In the preceding sections we illustrated the simple way in 
which transition network grammars can express some of Bever*s 
perceptual strategies. The transition network analyzes strings 
in essentially linear order, and the grammatical notation is 
flexible enough so that grammars can be devised to fit wide 
ranges of performance facts. However, to Justify the effort 
needed to simulate experimental data with network models, we must 
show that the resulting grammars offer substantial advantages 
compared to Informal verbal Interpretations, such as Bever » s . 

In this section we argue that these grammars are both conceptually 
and empirically productive: they lead to new theoretical questions, 

and they suggest new lines of experimentation, predicting specific 
outcomes. To the extent that the predictions of a particular 
grammar are confirmed, that grammar is validated as a model of 
the psychological processes Involved in sentence comprehension. 

The grammar shown in Figure 6, while only a small fragment 
of a complete English grammar, will suffice to exemplify the 
empirical implications of transition network models. It has been 
designed to account for the data underlying the perceptual 
strategies discussed above, but it also encompasses independent 
findings. The grammar mirrors the perceotual strategies Just so 
long as a depth- first search procedure is used to discover successful 
analysis paths. This search order implies that for truly ambiguous 
sentences, one interpretation will be recovered before the other; 
if required, the second Interpretation can be recovered by 
simulating a failure and continuing the analysis. This is in line 
with the results of MacKay and Bever ( 1967 ) and Foss et^. ( 1968 ): 
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MacKay and Bever found subjects to be aware that they arrived 
at one Interpretation of an ambiguous sentence first and could 
even report what the first Interpretation was. Foss et al . 
discovered that subjects tend to Interpret ambiguous sentences 
In only one wayi If the first Interpretation Is Incompatible 
with the experimental context, they can usually go on to find 
another Interpretation, although additional time Is required. 

The search strategy underlying the Figure 6 grammar accounts 
for these results even though the experiments are not Implicated 
In the perceptual strategies the grammar was designed to represent 

For ambiguous sentences within Its scope, the grammar 
clearly predicts which Interpretation should predominate. Other 
things being equal, the first Interpretation will have essentially 
the same analysis as the less complex of two unambiguous sentences 
with the same surface structure. Thus In a replication of 
the Foss et al . experiment, the first analysis of (l8a) should 
be the progressive, resembling (iMa), while the participial 
deep structure (iMb) should come out second. Subjects should 
first arrive at the Interpretation paraphrased In (18b), 
rather than (18a) ; 

(18) (a) They are flying planes. 

(b) Those people are piloting aircraft. 

(c) Those things are planes which are sailing 
through the air. 

The Figure 6 grammar similarly predicts the outcome for 
another class of ambiguous sentences, where a word can be analyzed 
either as the head of the subject noun-phrase, utilizing the 
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6 

prenomlnal adjective arc, or as the intransitive ' main verb (19-20) 
With the nounlike-ness markinf^s which permit the ambiguity, the 

(19) The Irish water boils, (a) Just like any other water. 

(b) but they still hurt. 

(20) The French bottle smells, (a) since it contained vinegar. 

(b) as well as Coca-Cola. 

grammar predicts that the interpretations corresponding to the 
(a) continuation will appear first. Thus again our simple grammar 
has concrete empirical implications. 

It should be noted that the ambiguities in (19-20) Involve 
a conflict betv/een two of Dover's nerceptual strategies. The (a) 
interpretations follow from the prenomlnal adjective strategy 
while the (b) interpretations are consistent with the functional 
labelling strategy, with water and bottle considered as the first 
verbs. Bever presents his strategies in isolation from one 
another, without specifying their interrelationships, but the 
transition network formalism requires the integration of all 
strategies into a single system. Potential strategy conflicts 
are highlighted, usually in the form of questions about the 
relative ordering of two or more arcs leaving a state, and the 
alternative grammar formulations often lead to the discovery of 
crucial cases that can be studied experimentally . Thus for 
example, the prenomlnal adjective and the functional labelling 
strategies could have been expressed in another grammar with 
the opposite precedence relation, so that the (b) Interpretations 
in (19-20) would be Judged less complex than the (a) ones. The 
choice between the Figure 6 grammar and this alternative model 
depends on the outcome of the ambiguity experiment described 
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above. 

Besides empirical consequences, questions of strategy 
interaction have important conceptual ramifications. Bever 
remarks that perceptual strategies express "generalizations which 
are not necessarily always true" ( Ibid . . p. 294, footnote 2), 
that each Isolated strategy will be misleading in some cases. 

The strategies thus serve as heuristic guidelines to the listener 
and do not directly reflect his abstract appreciation of the 
structure of his language. A transition network model, on the 
other hand, Incorporates a set of strategies and clarifies their 
interactions; the set as an Integrated whole is valid if it falls 
only for sentences which are truly unacceptable. Thus a transition 
network model Intended to make assertions about the listener's 
linguistic knowledge, whereas a set of isolated perceptual 
strategies is not. 

Transition network models raise other conceptual Issues: 
we have already- mentioned the question of selecting an appropriate 
complexity metric for the network, which is related to the problem 
of determining a small set of primitive, psychologically relevant 
actions and predicates . The network formalism also 

provides a new vocabulary for discussing the processes of language 
acquisition. We can imagine that as a child's linguistic 
abilities develop, a transition network model of his perceptual 
performance will evolve In stages 4f Increasing elaboration^ much 
as the grammar In Figure 6 grew out of Figure 1. New predicates 
and actions will appear, new arcs and states will be added, the 
order of arcs will be adjusted, and old and new arcs will 
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Interact to handle new syntactic constructions. It should be 
possible to demonstrate small, systematic deformations between 
the grammars representing the various levels of acquisition, and 
the sequence of grammars should have strong implications for 
models of adult performance. Finally, it is conceivable that 
detailed investigations of transition network acoulsitlon grammars 
will lead to an algorithm that simulates the language acquisition 
process, that takesothe kinds of data available to children at 
the different stages and devises appropriate perceptual models. 

VI. Conclusion 

The augmented recursive transition network we have described 
is a natural medium for expressing and explaining a wide variety 
of facts about the psychological processes of sentence comprehension. 
We have shown how several perceptual strategies can be represented, 
and in the last section we explored some of the empirical and 
conceptual Implications of these formalizations. These considera- 
tions illustrate the usefulness of transition network grammars 
as research tools and support their validity as perceptual models. 

Of course, there are several Important Issues we have not 
touched on: the role and representation of semantic information 

in sentence comprehension, the differences between the processes 
of sentence perception and production, and the correspondences 
between transition network grammars and conventional transfor- 
mational rules. We are currently investigating these problems. 

We are coupling the transition network parser to a semantic network 
so that non-syntactic features and context can guide the course 

of sentence analysis and lead to appropriate semantic Interpreta- 
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tions . We are also studying the formal and practical difficulties 
In using the transition network notation for writing generative 
grammars; we hope to find a simple algorithm for mapping adequate 
perceptual models Into equivalent production grammars. And 
finally, we are constructing two large transition network 
grammars, one based primarily on performance data and the other 
Intended to capture generalizations about linguistic competence 
as transformational grammars express them. We expect these 
grammars to converge, giving a single grammar and one notation 
for modelling both competence and performance. Reports on these 
Investigations are In preparation. 
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Footnotes 



1. The transition network parser was designed by William Woods. 

It Is programmed In BBN-LISP and Is currently running under 
the TENEX monitor system on a PDP-10 computer at Bolt, Beranek 
and Newman, Incorporated, Cambridge, Massachusetts. 

2. The parser Is presently being used as the natural-language 
front end of a system for accessing geological data on the 
Appollo lunar samples. 

3. Relative pronouns are also considered markers of subordinate 
clauses, so that Bever's strategy B would predict that relative 
clauses on subject noun-phrases should add more to perceptual 
complexity than the same clauses In posi:-verbal , object 
positions. If this Is true, relative clauses should not be 
Identified by a push within a noun-phrase, for this nredlcts 
the same degree of complexity for all relative clauses, no 
matter where they appear In relation to the main verb. We 

are currently exploring the possibility of analyzing relatives 
on subjects at the S level, by a system of arcs emanating 
from state S/SUBJ . This approach complicates the grammar to 
some extent, but It appears that It can account for the 
difficulty with subject relatives as well as the tremendous 
complexity of center-embedded sentences. It also explains 
the observation by Blumenthal (1966) that subjects tend to 
perceive canter-embedded constructions as simple structures 
of conjoined nouns and conjoined verbs. We will report 
on the details of this approach In the near future. 






li 



5. 




The problem with (l4b) Is not that perform Is intransitive, 
but that monkeys Is not permitted as Its object. Thus, It 
Is perfectly acceptable to say They are performing plays. 

A more precise grammar would allow the push at arc 3 and then, 
when the noun-phrase monkeys Is returned. It would be compared 
with the verb's object selectlonal restrictions and the analysis 
path would be aborted. We make the assumption that the verb 
Is Intransitive for simplicity of exposition. 

Intuitively, an adjective Is more "noun^lke" the more 
syntactic or semantic properties It shares with nouns. Thus 
large Is ranked below red and red belovj plastic because of 
the relative number of noun frames they can fit In, as In 
(a-d) : 

(a) I like red. 

(b) *I like large. 

(c) The toothbrush Is made of plastic. 

(d) *The toothbrush Is made of red. 
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6. Examples which are ambiguous ortho^raphlcall^ , such as 

(19-20), are more difficult to discover and seem more strained 
than acoustically ambiguous ones: 

(a) (1) The sun’s rays meet. 

(11) The sons raise meat. 

(b) (1) The producer’s show flops. 

(11) The producers show flops. 

I am Indebted to John Ross and Michael Maratsos for these 
examples. 
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Figure 1 

A Simple Transition Network Grammar 





Arc Condition Actions 

1 T (SETR SUBJ *) 

2 (AND (GETF TNS) (SETR TNS (GETF TNS)) 

(SVAGR SUBJ 

(GETF PNCODE))) 

3 (TRATJS V) (SETR OBJ *) 

A (INTRANS V) 

5 T 

6 T 

7 T 



(SETR DET *) 
(SETR N *) 



Figure 2 

Trace of an Analysis 




Sentence: The man kicked the ball. 



STKINf’ = (THF MAN KICKF.O THF! FALL) 
FNTFhlNG STA'IF S/ 

ABOUT 10 HUSH 
FNlFklNG STAIF NP/ 

TAKING CAT DFT AhC 

STkING = (MAN KICKED THE HALL) 
FNIFKING STATE NP/DET 
TAKING CAT N AkC 

STkING = (KICKED THF PALL) 
FNTFkING STATE NF/N 
ABOUT TO HOP 
FNlFklNG STATE S/SlJPJ 
TAKING CAT V AkC 



O 

ERIC 



STUNG = (THE FALL) 

FNTEkING STATE VP/V 
STOKING ALTAhC ALTFKNATIVE 76869 
ABOUT TO PUSH 
FNTEkING STATE NP/ 

TAKING CAT DET AkC 

STkING = (HALL) 

FNTEKING STATE NP/DET 
TAKING CAT N AkC 

STKING = NIL 
FNTEKING STATE NP/N 
ABOUT TO POP 
FNTEkING STATE S/VP 
ABOUT TO POP 
SUCCESS 

10 ARC ATTEMPTED 
19b CONS'ES 
1.8869999 SECONDS 
PAKSINGS: 

S NP DET THE 
N MAN 

AUX TNS PAST 
VP V KICK 

NP DET THE 
N BALL 



(The alternative analysis path 
starting with arc ^ is saved) 



(Number of memory words used) 
(Processing time required) 

(The recovered deep structure) 
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Figure 3 

Arcs Required for Passives 




Arc Condition Actions 



4 (OR (INTRANS V) 

(PULLR OBJ)) 

5 (PULLR SUBJ) 



9 


(AND (CETF PASTPART) 
(PASSIVE *) 
(WRD BE V)) 


(SETR OBJ (SUBJ) 
(SETR SUBJ NIL) 
(SETR V *) 


10 


(NULLR SUBJ); 




U 


T 


(SETR SUBJ *) 


12 


(NULLR SUBJ) ' 


(SETR SUBJ 

(BUILDQ (NP (PRO SOMEONE)))) 
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Figure 

A Strategy for Clausal Relationships 
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13 


(NULLR SUBORD) 


(SETR SUBORD *) 


14 


(NULLR SUBORD) 


(SETR SUBORD *) 


15 


(CAT RELPRO) 


(SENDR WU (NPBUILD)) 
(ADDR REL *) 


16 


T 


(SENDR WH (NPBUILD)) 
(ADDR REL *) 


17 


T 





ERIC 
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Figure 5 

Progressive and Prenominal Participle Arcs 
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Condition 


Actions 
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(AND (GETF PRESPART) 
(WRD BE V)) 


(SETR V *) 

(ADDR TNS (OUOTE PROGRESS)) 


19 


(GETF PRESPART) 


(ADDR NI^DS *) 



Figure 6 

Prenominal Adjectives 
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(GE (NLIKE •) 


(ADDR NMODS N) 




(NLIKE N)) 


(SETR N *) 



o 

ERIC 
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